Skip to content

[test optimization] [SDTEST-1355] Fix DI issues with auto test retries#5072

Merged
juan-fernandez merged 14 commits intomasterfrom
juan-fernandez/fix-di-atr-issues
Jan 14, 2025
Merged

[test optimization] [SDTEST-1355] Fix DI issues with auto test retries#5072
juan-fernandez merged 14 commits intomasterfrom
juan-fernandez/fix-di-atr-issues

Conversation

@juan-fernandez
Copy link
Copy Markdown
Collaborator

@juan-fernandez juan-fernandez commented Jan 7, 2025

What does this PR do?

Fix different issues with the integration between Dynamic Instrumentation and Test Optimization.

Motivation

  • Line probes were not cleaned up after the test finished. This caused an issue where we tried to add a line probe that was already there.
  • We did not wait for hit probes to return their information to the main process. This caused a race condition where we assigned the snapshot to the wrong test.
  • We were working under the assumption that the line probe would always be located at the top of the test error stack trace. This is not true since the user code might be at a different place.

Unrelated to this PR: I fixed the way we reported TestingLibraryElementError. Instead of showing the error message in the stack, we clean up the stack.

Plugin Checklist

  • Unit tests.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jan 7, 2025

Overall package size

Self size: 8.39 MB
Deduped: 94.74 MB
No deduping: 95.25 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.3.0 | 29.43 MB | 29.43 MB | | @datadog/native-appsec | 8.4.0 | 19.25 MB | 19.26 MB | | @datadog/native-iast-taint-tracking | 3.2.0 | 13.9 MB | 13.91 MB | | @datadog/pprof | 5.4.1 | 9.76 MB | 10.13 MB | | protobufjs | 7.2.5 | 2.77 MB | 5.16 MB | | @datadog/native-iast-rewriter | 2.6.1 | 2.59 MB | 2.73 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 3.1.0 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 826.22 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | semver | 7.6.3 | 95.82 kB | 95.82 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.1 | 51.46 kB | 51.46 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@juan-fernandez juan-fernandez changed the title [test optimization] Fix DI issues with auto test retries [test optimization] [SDTEST-1355] Fix DI issues with auto test retries Jan 7, 2025
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Jan 7, 2025

Benchmarks

Benchmark execution time: 2025-01-14 10:38:29

Comparing candidate commit 6b2a867 in PR branch juan-fernandez/fix-di-atr-issues with baseline commit b36ce05 in branch master.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 772 metrics, 25 unstable metrics.

scenario:log-without-log-20

  • 🟥 cpu_user_time [+18.837ms; +23.991ms] or [+5.482%; +6.982%]

@juan-fernandez juan-fernandez force-pushed the juan-fernandez/fix-di-atr-issues branch from a0b4510 to 6ad15c3 Compare January 8, 2025 15:41
@codecov
Copy link
Copy Markdown

codecov Bot commented Jan 8, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 61.54%. Comparing base (b36ce05) to head (6b2a867).
Report is 14 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #5072       +/-   ##
===========================================
- Coverage   88.97%   61.54%   -27.44%     
===========================================
  Files          11      111      +100     
  Lines         635     3791     +3156     
===========================================
+ Hits          565     2333     +1768     
- Misses         70     1458     +1388     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@juan-fernandez juan-fernandez marked this pull request as ready for review January 10, 2025 15:51
@juan-fernandez juan-fernandez requested review from a team as code owners January 10, 2025 15:51
}).unref()

this.worker.on('error', (err) => log.error('ci-visibility DI worker error', err))
this.worker.on('messageerror', (err) => log.error('ci-visibility DI worker messageerror', err))
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was already handled above

watson
watson previously approved these changes Jan 14, 2025
Copy link
Copy Markdown
Collaborator

@watson watson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed changes to the following files only:

  • packages/dd-trace/src/ci-visibility/dynamic-instrumentation/index.js
  • packages/dd-trace/src/ci-visibility/dynamic-instrumentation/worker/index.js

I have only a small question (see comment), but besides that it look good 👍

Comment thread packages/dd-trace/src/ci-visibility/dynamic-instrumentation/worker/index.js Outdated
Co-authored-by: Thomas Watson <w@tson.dk>
@juan-fernandez juan-fernandez requested a review from watson January 14, 2025 10:28
@juan-fernandez juan-fernandez merged commit 71c4307 into master Jan 14, 2025
@juan-fernandez juan-fernandez deleted the juan-fernandez/fix-di-atr-issues branch January 14, 2025 10:43
watson added a commit that referenced this pull request Jan 22, 2025
@watson watson mentioned this pull request Jan 22, 2025
watson added a commit that referenced this pull request Jan 22, 2025
@watson watson mentioned this pull request Jan 22, 2025
watson added a commit that referenced this pull request Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants